
BACKGROUND OF THE INVENTION 
Field of the Inven-bion 



5 The present invention relates to an image pickup 

method and an image pickup apparatus capable of 
broadening an apparent dynamic range of a video signal . 
Related Background Art 

Image pickup devices are used widely as an 

10 image pickup unit of VTRs with a built-in camera, still 
video cameras and the like. An image pickup apparatus 
using an image pickup tube of a video camera or a solid 
image pickup element has a dynamic range narrower than 
that of silver-salt photographing systems, and is 

15 therefore associated with some problems of "crushed 

white" with lost tonal levels in a bright portion and 
"crushed black" with lost tonal levels in a dark 
portion, under rear light or in other cases. 

If such a case occurs with, for example, a video 

20 camera, a user increases an exposure amount of a main 

object with "crushed black" by manually controlling the 
iris or operating upon a rear light correction button 
to open the iris by about two steps. 

Even if a proper exposure amount is obtained for . 

25 the main object through rear light correction, the 

background has "crushed white" and becomes only white. 
Therefore, with a conventional image pickup apparatus. 
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proper adjustment only of the exposure amount for a 
main object does not solve a problem of a narrow 
dynamic range of the image pickup apparatus. 

In order to solve the problem of "crushed white" 
5 and "crushed black", in an image pickup apparatus which 
uses a line scanner or the like for the conversion of a 
still image into electrical signals, a single image is 
synthesized from a plurality of images of the same 
object picked up at different exposure amounts. 

10 In the case of such an image pickup apparatus 

taking only a still image, the positional relation 
between an object and the apparatus is fixed. 
Therefore, there is no shift between images although 
they are picked up at different timings, so that there 

15 is no practical problem even if a single image is 
synthesized from a plurality of images. 

However, in the case of an image pickup apparatus 
such as a video camera or a still video camera, the 
above-described image pickup method cannot be applied 

20 reliably because the positional relation between an 

object and the apparatus may change minutely because of 
hand vibration or the like. 

In such a case, the position of each of a 
plurality of images of the same object picked up at 

25 different timings has a small shift. Therefore, a 
single image synthesized from these images has the 
object with an image shift such as double or triple 
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objects. This becomes a serious issue for recent 
compact and light video cameras which are likely to 
have hand vibration. 

SUMMARY OF THE INVENTION 

n consideration of the above-described problems, 
t of the present invention is to provide an 



ff^ an obj 



image pickup method and an image pickup apparatus 
capable of picking up an image with a broad substantial 

10 dynamic range\without the image shift, even with video 
cameras or stiri video cameras which are likely to be 
subject to image sshifts. 

According to one aspect of the present invention, 
there is provided an image pickup method of increasing 

15 an apparent dynamic range of a video signal by 

synthesizing a single image from a plurality of images 
sequentially picked up at different exposure amounts, 
wherein a motion vector of a video signal is detected, 
and if the detection result indicates that the motion 

20 vector is larger than a predetermined threshold value, 
image synthesization is not performed. 

According to another aspect of the present 
invention, there is provided an image pickup method of 
increasing an apparent dynamic range of a video signal 

25 by synthesizing a single image from a plurality of 
images sequentially picked up at different exposure 
amounts, comprising: a motion vector detecting step of 
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detecting a motion vector between corresponding pixels 
in the plurality of images; a vector difference 
detecting step of detecting a difference between a 
motion vector detected at the motion vector detecting 
step and a motion vector between the plurality of 
images; a comparison step of comparing the detection 
result of the vector difference detecting step with a 
predetermined threshold value; and a synthesization 
inhibiting step of inhibiting the image synthesization 
if the comparison result at the motion vector 
comparison step is larger than the predetermined 
threshold value. 

According to another aspect of the present 
invention, there is provided an image pickup method of 
increasing an apparent dynamic range of a video signal 
by synthesizing a single image from a plurality of 
images sequentially picked up at different exposure 
amounts, comprising: a motion vector detecting step of 
detecting a motion vector between corresponding pixels 
in the plurality of images; a vector difference 
detecting step of detecting a difference between a 
motion vector detected at the motion vector detecting 
step and a motion vector between the plurality of 
images; a comparison step of comparing the detection 
result of the vector difference detecting step with a 
predetermined threshold value; a coordinate converting 
step of performing a coordinate conversion of the 
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plurality of images in correspondence with an image 
shift caused by a time difference when the plurality of 
images are picked up, if the comparison result at the 
motion vector comparison step is smaller than the 
5 predetermined threshold value; and an image 

synthesizing step of synthesizing the plurality of 
images with the image shift corrected at the coordinate 
converting step, into a single image. 

According to another aspect of the invention, 
Q 10 there is provided an image pickup apparatus for 

^2 increasing an apparent dynamic range of a video signal 

71 by synthesizing a single image from a plurality of 

images sequentially picked up at different exposure 
amounts, comprising: motion vector detecting means for 
^ 15 detecting a motion vector of the video signal; 

lU comparison means for comparing the detection result by 

^fl the vector difference detecting means with a 

rfi 

predetermined threshold value; and synthesization 

inhibiting means for inhibiting the image 
20 synthesization if the comparison result by the 

comparison means is larger than the predetermined 

threshold value. 

According to another aspect of the invention, 

there is provided an image pickup apparatus for 
25 increasing an apparent dynamic range of a video signal 

by synthesizing a single image from a plurality of 

images sequentially picked up at different exposure 
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amounts, comprising: motion vector detecting means for 
detecting a motion vector between corresponding pixels 
in the plurality of images; vector difference detecting 
means for detecting a difference between a motion 
5 vector detected by the motion vector detecting means 
and a motion vector between the plurality of images; 
comparison means for comparing the detection result by 
the vector difference detecting means with a 
predetermined threshold value; coordinate converting 

10 means for performing a coordinate conversion of the 
plurality of images in correspondence with an image 
shift caused by a time difference when the plurality of 
images are picked up, if the comparison result by the 
motion vector comparison means is smaller than the 

15 predetermined threshold value; and image synthesizing 

means for synthesizing the plurality of images with the 
image shift corrected by the coordinate converting 
means, into a single image. 

The invention provides the above-described 

20 technical aspects. Therefore, in increasing an 
apparent dynamic range of a video signal by 
synthesizing a single image from a plurality of images 
sequentially picked up at different exposure amounts, 
if a difference between a motion vector between pixels 

25 in a plurality of images and a motion vector between 

images is larger than a predetermined threshold value, 
the pixel is not used for synthesization. Even a video 




signal having an image shift: caused by a motion of an 
object itself or hand vibration can be prevented from 
forming a double or triple images of an object. 

Other objects, features and advantages of the 
5 invention will become apparent from the following 
detailed description taken in conjunction with the 
accompanying drawings . 

BRIEF DESCRIPTION OF THE DRAWINGS 
10 Fig. 1 is a block diagram showing the structure of 

Q a VTR with a built-in camera embodying the present 

invention. 

Fig. 2 is a block diagram showing the specific 
structure of a camera unit of VTR shown in Fig. 1. 
]3 15 Fig. 3 is a timing chart illustrating an operation 

fy of an image pickup device. 

C= Fig. 4 is a diagram explaining a concept of image 

processing according to an embodiment of the invention. 
Fig. 5 is a block diagram showing the specific 
20 structure of an operation (computing) circuit of VTR 
shown in Fig . 1 . 

Figs. 6A and 6B are diagrams explaining a method 
of determining threshold levels to be used for the 
discrimination between "crushed white" and "crushed 
25 black". 

Figs, 7A, 7B, 7C and 7D are diagrams explaining 
tonal level characteristics. 



Fig. 8 is a timing chart illustrating the 
operation of an image pickup device. 

Fig. 9 is a block diagram showing the partial 
structure of the camera unit . 

Fig. 10 is a block diagram showing an example of a 
control circuit of VTR shown in Fig. 1. 

Figs. IIA, IIB and llC are diagrams explaining a 
concept of image shifts according to another 
embodiment . 

F^g. 12 is a block diagram showing the specific 
structurfe of VTR of the embodiment. 

ig. is a timing chart illustrating the 

operation ofi VTR of the embodiment. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

An image pickup method and an image pickup 
apparatus according to preferred embodiments will be 
described in detail with reference to the accompanying 
drawings . 

Fig. 1 is a block diagram showing the overall 
structure of a VTR with a built-in camera embodying the 
present invention . 

In Fig. 1, reference numeral 100 denotes a camera 
unit, reference numeral 200 denotes a processing unit, 
and reference numeral 300 denotes a recording unit. In 
the camera unit 100, light incident upon an optical 
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system 101 is controlled in amount by an iris 102 and 
focussed on an image pickup device (element) 103. 

The image pickup device 103 is made of 
semiconductor elements such as MOSs and CCDs. An 
5 optical image focussed on the focal plane of the image 
pickup device 103 is converted into electrical video 
signals to generate a video signal to be output. 

A focus drive circuit 107, an iris drive circuit 
106 and an image pickup device drive circuit 105 for 
O 10 controlling a light accumulation time or the like drive 

O the optical system 101, iris 102, and image pickup 

|1 device 103, under the control of a camera control 

m circuit 108. A camera signal processing circuit 104 

' performs a gamma correction process or other known 

15 processes, similar to a signal processing circuit of a 
If, general video camera . 

Phe video signal output from the camera unit 100 
is an ai%alog signal which is supplied to the processing 
unit 100 ahd converted into a digital signal by an A/D 
20 converter 20rVv Pixel data of the converted digital 
signal is then c^>nverted by an operation (computing) 
circuit 202 in a maimer to be described later. 
Thereafter, the converted pixel data is converted back 
into an analog signal by\a D/A converter 203 and 
25 supplied to the recording unit 300. 

Reference numeral 204 denotes an image memory 
which is used by the operation circuit 202 and a motion 




vector detection circuit: 206 for data calculation. 
Reference numeral 205 denotes a memory control circuit 
for controlling the operation of the image memory 204. 
The memory control circuit 205 outputs read/write 
address signals of the image memory 204 in response to 
a timing signal supplied from the camera control 
circuit 108 of the camera unit 100. 

The memory control circuit 205 also outputs an 
address signal for frame position alignment, according 
to an output from the motion vector detection circuit 
206, when images are synthesized by correcting an 
image shift. 

A motion vector comparison circuit 207 compares a 
motion vector of each pixel detected by the motion 
vector detection circuit ^QS' with a motion vector 
between images calculated from the motion vector of 
each pixel. If a difference of the motion vector of 
each pixel from the motion vector between images is 
equal to or larger than a predetermined threshold 
value, position information of each pixel is supplied 
to the operat ion cire uit_2P.2.^to_Jj^ synthesiz ation 
of pixels. 

The operation circuit 202 synthesizes read common 
signals and performs enlarging/interpolation in order 
to convert the common signals into signals of the same 
type as original input signals. In this manner, the 
operation circuit 202 outputs a video signal of even a 
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moving object;, with a synthesized tonal level without 
an image shift. 

The video signal output from the operation circuit 
202 is converted by the D/A converter 203 into an 
analog signal and recorded in VTR (video tape recorder) 
301. If VTR 301 is a digital recording type, the D/A 
converter 203 is not necessary. 

ixt, the details of the operation of the image 
pickup element 103 will be described. Fig. 2 is a 
block diagram showing the more detailed structure of 
the camera unit 100, and Fig. 3 is a timing chart 
illustrating the operation of the camera unit 100 
assuming tha^ NTSC signals are used. 

Referring to Fig. 3, a field index signal ( Fl ) is 
a signal used for discriminating between odd and even 
fields constituting one frame. 

A VgLK signal is a vertical blanking signal, the 
period with H (high) of this signal corresponding to an 
effective frame and the period L (low) corresponding to 
a vertical blanking period. 

A Tp^^^gg signal controls a charge accumulation time 
of the image pickup device 103. This Tp^^^^ signal is a 
pulse signal used for reading a pixel to be output to a 
vertical transfer CCD, if the image pickup device is a 
CCD image pickup device. 

An iris gate signal is a signal designating which 
of a 1/1000 sec. accumulation signal and a 1/60 sec. 
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accumulation signal is used as a standard video signal 
for au-tomatic exposure to be described later. 

n the example shown in Fig. 3, during the first 



f^blankiAg period, the 1/1000 sec. charge accumulation is 
5 performe)a, and during the next effective period, the 

1/1000 seo^. accumulation signal is output. Immediately 
after the l\lOOO sec. accumulation period, the 
substantial 60 sec. charge accumulation is performed, 
and during theViext field effective period, the 1/60 
10 sec. accumulatioti signal is output. In this manner, 
signals of difference light amounts (1/1000 sec. and 
1/60 sec. ) are alternately output for each field. 

In Fig. 2, reference numeral 20 denotes a known AE 
control circuit which receives a signal (e.g., video 
15 signal) from the camera signal processing circuit 104 

via the A/D converter 201 and calculates and outputs an 
^3 exposure control signal. Reference numeral 22 denotes 

a known AF control circuit which outputs a focus 
control signal. Reference numeral 24 denotes a 1/2 
20 frequency division circuit for lowering the frequency 
of the vertical blanking signal Vb^k ^ half. 

Reference numerals 26 and 27 denote sample-hold 
circuits, reference numeral 28 denotes an inverter, and 
reference numerals 29 and 30 denote switches which 
25 select either an output of the 1/2 frequency division 
circuit 24 or an output of the 1/2 frequency division 
circuit 24 inverted by the inverter 28, in order to 
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determine a sampling timing. Outputs of the sample- 
hold circuits 26 and 27 are applied to the iris drive 
circuit 106 and focus drive circuit 107, respectively, 
to thereby perform an automatic exposure control, an 
5 automatic focus control and the like according to those 
outputs . 

In this embodiment, a combination of 1/1000 sec. 
and 1/60 sec. is used, which corresponds to a light 
amount change of about four steps ( sixteenf old ) . 
10 Therefore, for example, in the case of a camera using a 
CCD image pickup device, for example, if an exposure 
control of a main object is performed by using as a 
reference the 1/60 sec. accumulation time in the even 
field, it happens frequently that the background of the 
15 even field is likely to have the "crushed white", 

whereas the main object in the odd field with the light 
amount lowered by four steps is likely to have the 
"crushed black". This example assumes the case wherein 
proper exposure is set to the background for rear light 
20 correction, and in accordance with the scene at that 
time, the accumulation time may be set to a value 
different from 1/1000 sec. 

different from the embodiment CCD, a CCD having 
a high speed shutter function of a VOD type has been 
25 recently used. This type drains unnecessary charges in 
the vertical d^ection of a CCD substrate, and can set 
the shutter speedVvery finely. If this CCD is used. 
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tne camera drive circuit 105 can set an optimum shutter 
speVd which is judged by the AE control circuit 20 of 
the control circuit 108 in accordance with a brightness 
difference between a main object and a background 
object . 

As described earlier, in this embodiment, the 
image quality is improved by positively utilizing the 
"crushed white" and "crushed black" in each field. 
That is, a portion with the "crushed white" or "crushed 
black" is replaced by a corresponding portion in the 
other field (since exposure is different, the "crushed 
black" or "crushed white" is not being formed in the 
other field), and video signals of both the fields are 
synthesized to obtain a final video signal. 

The above fundamental approach will be explained 
with reference to Fig. 4, In Fig. 4, a main object is 
schematically shown as a rectangle with a vertically 
long side. A through image (T) shown in Fig. 4 
corresponds to an image formed directly by an output of 
the image pickup device 103. A memory image (M) 
corresponds to an image formed by a preceding field 
image signal once stored in the image memory 204, The 
through image has the main object with the "crushed 
black" by rear light at each odd field, whereas it has 
the background with the "crushed white" at each even 
field. 
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ince the memory image is formed by a signal 
delayea\by one field, the "crushed white" and "crushed 
black" ar&». formed in the fields different from those of 
the through N^mages . Therefore, if the through and 
memory images ^^re properly combined, an image of a good 
quality without Nthe "crushed white" and "crushed black" 
can be obtained. \Iamely, signals of the through and 
memory images are coSEnpared with predetermined threshold 
values at each field tb discriminate between the 
10 "crushed white" and "crii^hed black" of each pixel, by 
setting "1" if the signal ds^jf each pixel is larger than 
the threshold value, and "0"\if smaller. 

igs. 6A and 6B show the relation between 
^hreshoi^^ values, pixel brightness levels, and fields. 
15 The abscis^ of Fig. 6A indicates the brightness level, 
and the ordii^iate indicates an occurrence frequency of 
each brightnessxLevel in one frame. As shown in Fig. 
6A, a first threshold value Thl is set so that the 
"crushed black" can discriminated, whereas a second 
20 threshold value Th2 is set so that the "crushed white" 
can be discriminated. Thdsf is, a brightness level 
equal to or higher than theVirst threshold value Thl 
is judged as the "crushed black", whereas a brightness 
level equal to or higher than the second threshold 
25 value Th2 is judged as the "crushelt white" . 

Fig. 6B shows the relation between each field and 
a corresponding threshold value. Since the "crushed 
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white" and "crushed black" appear alternately in the 
odd and even fields, the threshold values used for the 
discrimination are also changed for each field . 

It is possible therefore to judge which pixel in 
5 which field has the "crushed black" or "crushed white". 
By using the judgement results, a pixel signal having a 
proper exposure amount can be selected from either the 
through or memory image. For example, in the odd 
field, the through image is selected for a pixel having 

S 10 a logical product "1" of a judgement A and a judgement 

33 B, and the memory image is selected for a pixel having 

the logical product "0". In the even field, the 

ijl opposite relation as above is set. In this manner, 

selection flags shown in Fig. 4 can be obtained. 

m 15 Images synthesized in accordance with the 

I il 

Lj selection flags are shown at the bottom row of Fig. 4. 

As shown, moving images sufficient for practical use 
can be obtained, by suppressing the influence of the 
time axis shift of each image assuming that the main 

20 object is moving at a constant speed. 

Fig. 5 is a block diagram showing the details of a 
circuit portion in the operation circuit 202 of the 
processing circuit 200, the circuit portion being used 
for the comparison with the first and second threshold 

25 values Thl and Th2 and for the generation of the 

selection flags. A threshold value switch control 
signal alternately changes between "H" and "L" at each 




field, similar to an FI signal. The threshold value 
switch control signal is applied to a first threshold 
value generation circuit 53, and via an inverter 51 to 
a second threshold value generation circuit 52. 
5 In accordance with the threshold value switch 

control signal, the first and second threshold value 
generation circuits generate the first or second 
threshold value Thl or Th2 having the relation shown in 
Fig. 6B. Comparison circuits 54 and 55 compare the 
Q 10 memory and through images with the threshold values 

O supplied from the threshold value generation circuits 

1=2^ 52 and 53, and output judgement signals A and B. 

Ill An AND gate 56 outputs a selection flag signal as 

' the logical product of the judgement signals A and B. 

m 15 A switch 57 selects either the memory or through 

lJI image in accordance with the selection flag signal. 

:^ Figs. 7A to 7D are diagrams explaining tonal level 

characteristics. A solid line in Fig. 7A shows a 
characteristic of a general video camera. Up to a 100 
20 % input, the input/output characteristic is linear. 

For an input (100 to 400 %), a gentle slope called a 
knee characteristic appears. A transition point 
between the linear characteristic portion and knee 
characteristic portion is represented by PI. For a 
25 high speed shutter, this transition point moves to P2 . 

It is assumed that PI is 1/60 sec and P2 is 1/250 sec. 
with an exposure change of two steps. 
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As described earlier, if the exposure amount is 
four steps, the shutter speed change is represented by 
the relation between (1) and (5) shown in Fig. 7D. (1) 
in Fig, 7D corresponds to 1/60 sec, (2) corresponds to 
1/125 sec, (3) corresponds to 1/250 sec, (4) 
corresponds to 1/500 sec, and (5) corresponds to 
1/1000 sec A characteristic with a desired curve can 
be obtained by using two characteristic lines having 
di f f erent slopes • 

Fig, 7B shows examples of synthesized 
characteristic lines in which different threshold 
values are used for the discrimination between the 
"crushed white" and "crushed black". 

Fig. 7C shows characteristic lines, (1) outputting 
an addition average of corresponding two pixel signals, 

(2) outputting a selected one of two pixel signals, and 

( 3 ) outputting an addition average with predetermined 
coefficients . 

In the above example, a time resolution is 
substantially 30 images per second which is almost 
equal to a frame storage CCD image pickup device. In 
order to realize a time resolution almost equal to a 
field storage CCD image pickup device, an example of 
picking up two images in one field, will be described. 
A changed circuit portion is shown in Fig. 9, and the 
timing chart is shown in Fig. 8. 
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Principally, signals are read from the image 
pickup device 103 at: a speed higher than a general 
video rate, and the read signals are converted in a 
time axis into a normal rate. First and second field 
memories 90 and 91 each have a memory capacity of one 
field image information. The first field memory 90 
delays a 1/1000 sec. accumulation signal in order to 
synchronize it with a 1/120 sec. read signal. The 
second field memory 91 performs a twofold time 
expansion in order to change the 1/120 sec. video 
signal to an NTSC signal of 1/60 sec. 
r- I J^^^ows (a) to (d) shown in Fig. 9 correspond to 
■^signalsV ( a) to (d) shown in Fig. 8. The image pickup 
device lo!S^ capable of performing the above operation 
may be a MOSw solid image pickup device of an XY 
addressing type. 

Next, another example of the details of the 
control circuit 108 will be described with reference to 
Fig- 10. A master clock generator 40 generates an 
internal master clock to be used by the control circuit 
108, in accordance with an external reference signal - 

A clock generator 41 for a 1/1000 sec. shutter 
generates a high speed clock from the master clock 
supplied from the master clock generator 40. A clock 
generator 42 for a 1/60 sec. shutter generates a low 
speed clock from the master clock supplied from the 
master clock generator 40. 
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A switch 45 changes its contact point alternately 
at each field to alternately apply the outputs of the 
clock generators 41 and 42 to the drive circuit 105. 

An AE control signal generator 43 generates an AE 
control signal for the iris control, in accordance with 
the video signal supplied from the camera signal 
processing circuit 104. A control signal holding 
circuit 44 holds the AE control signal during one field 
period . 

10 x^^^^^^^^^P^ switch 46 changes its contact point alternately 
l^^^t eacfe field to alternately apply the AE control 

signal of the AE control signal generator 43 and the AE 
control siSanal held by the control signal holding 
circuit, toV;he iris control circuit 106. A switch 
15 signal genera^r 47 controls the switching of the 

switches 45 and\46 which are synchronously switched. 

the above example, the clock generators for 
^'^^generatrng high and low speed clocks are provided, and 
the high ahd low speed clocks are switched in response 
20 to a signal of the switch signal generator output at 
each field. Accordingly, the circuit structure and 
operation can be simplified which are particularly 
suitable for moving i^mages. 

In the above description, images are formed at 
25 different exposure amounts by changing the shutter 

speed. If a high speed iris function is available, the 
iris may be changed at high speed. Alternately, a 
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light reducing filter using a spatial light modulating 
device such as PLZT may be electrically controlled. 

As described above, various calculations may be 
made in this embodiment. An example of image 
5 synthesization to be performed with a simple process 

and more effective simple addition calculation will be 
described. 

Figs. IIA to lie are conceptual diagrams 
explaining an image shift of an object in the two 
10 images (odd and even field images) relative to a 
display screen (image pickup device 103). 

It is assumed that the upper left in the common 
area of the even and odd images is a memory read start 
point P. Then, a point Pouu shown in Fig. IIB is the 
15 image memory read start point for the odd image, and a 
point Peven sl^own in Fig. IIC is the image memory read 
start point for the even image. By using these start 
points, video signals with no image shift can be 
generated for the common area of the even and odd 
20 images. 

A specific example of the structure shown in Fig. 
1 applied to this embodiment is shown in Fig. 12. 

Referring to Fig. 12, an input video signal is 
converted into a digital signal by the A/D converter 
25 201, and supplied to a first field memory 204a and a 
motion detection circuit 206. 
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An output of the first field memory 204a is 
supplied to another input terminal of the motion vector 
detection circuit 206. The motion vector detection 
circuit 206 detects motion information of each pixel 
5 from two video signals having one field time delay, and 
in accordance with this motion information, generates 
an image shift correction signal. This image shift 
correction signal is supplied to the memory control 
circuit 205 which controls the first field memory 204a 
Q 10 in accordance with the image shift correction signal. 

5 The image shift correction signal generated by the 

motion vector detection circuit 206 is supplied also to 
the motion vector comparison circuit 207 . The motion 
""^ vector comparison circuit 207 is constituted of a 

15 threshold generation circuit 61, a comparison circuit 
ry 62, an address conversion circuit 63, and the like. 

?0 The image shift information output from the motion 

vector detection circuit 206 is supplied to the 
threshold generation circuit 61 and comparison circuit 
20 62. The threshold generation circuit 61 generates a 
threshold value to be used for the judgement of an 
object, and supplies it to the comparison circuit 62. 
The comparison circuit compares the threshold 



value with the motion information of each pixel to 



25 judge whether the motion of eac h pixel is caused by 
hand vibration or by some other motion. In other 



words, it is judged whether the motion of each pixel is 
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caused by the mot:ion of the object itself, and an 
address of a pixel which was judged as having a motion 
other than by hand vibration, is supplied to the 
address conversion circuit 63, 

In accordance with interpolation information 
supplied from an enlarging/interpolation circuit 202a, 
the address conversion circuit 63 converts an address 
of the pixel which was judged as having a motion other 
than by hand vibration, into an address after 
enlarging/interpolation, generates a selection signal 
for inhibiting the pixel to be synthesized and 
selecting the pixel in the field with a proper 
exposure, and supplies the selection signal to a next 
stage selector 202c. 

Rhe video signal of the common area shown in Fig. 
"11 A is Vutput from the first field memory 204a and 
convertec\ into a signal of the original type (e.g., 
NTSC signaik) same as the input signal. 

The smaller the common area, the larger the 
enlarging factor becomes. Regardless of what enlarging 
factor is used, the video signal is converted into a 
signal of the original type and stored via an adder 
202b into a second field memory 204b. 

Assuming that the video signal stored in the 
second field memory 204b is of the odd field, the video 
signal of the next even field is output from the 
enlarging/interpolation circuit 202a and added to the 
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corresponding video signal by the adder 202b for each 
pixel . 

[■he addition signal is supplied via the terminal A 
of the ^elector 202c and via the EVEN terminal of 
another selector 207 to the D/A converter 203. The 
addition signal is also supplied via the terminal A of 
the selectoA202c to the second field memory 204b. 
During the nex^ odd field, new odd field information is 
stored in the second field memory 204b while the same 
video signal is sOx)plied to the D/A converter 203 via 
the ODD terminal oAthe selector 207 through a read- 
modify- write operatic 

switch 208 is open for the odd field and close 



ft^jl^or theXeven field. While the new odd field 
15 informati\^n is written in the second field memory, the 
adder 202b\is made through by connecting to the 
terminal E. 

'j^^ll^^A^^ A of the selector 202c is supplied 

with tl^fe addition signal, the terminal O is supplied 
with the \dd field information output from the second 
field memor\ 204b, and the terminal E is supplied with 
the even fieldV information. If the selection signal 
from the motion \ector comparison circuit 207 indicates 
that a pixel has aNpiotion caused by hand vibration, the 
25 addition signal is sheeted, whereas if the selection 
signal indicates that ay pixel has a motion other than 
by hand vibration, a vidGKD signal of either the even or 
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oda field having a proper exposure is selected. The 
selec^d signal is supplied to the next stage selector 
207 and second field memory 204b. A video signal 
generated ria the above manner is output from the D/A 
converter 203\in the form of analog signal same as the 
original input svLgnal . 

The operation of the above-described processes is 
illustrated in the timing chart of Fig. 13. 

In the timing chart shown in Fig. 13, a signal 
indicated at A is a discrimination signal between the 
odd and even fields and corresponds to an output timing 
of the enlarging/interpolation circuit 202a. 

A signal indicated at B shows an open/close timing 
of the switch 208 . The function of the adder 202b 
changes with this open/close of the switch 208. If the 
switch 208 is open, the signal output from the 
enlarging/ interpolation circuit 202a passes through the 
adder 202b, whereas if the switch 208 is close, the 
signal is added by the adder 202b. 

A signal indicated at C shows a write timing of 
the second field memory 204b. During the odd field 
indicated by the signal A, the odd field image signal 
is written in the second field memory, whereas during 
the even field indicated by the signal A, the video 
signal selected by the selector 202c according to the 
read-modif y-write operation is written again in the 
second field memory 204b at the same addresses. 



# # 

- 26 - 



jffl\frK y\ A signal Indicated at D shows a switching timing 
of -thfe selector 207. Image informat:ion synthesized 
from t^ odd field image information during a period tl 
and the ^ven field image information during a period t2 
is outputXduring a period t2, and the same synthesized 
image inforwnation is again read and output during a 
period t3 . 

n synthesizing an image, if it is judged that a 
h^s a motion other than by hand vibration, a 
video sigi^^l of either the even or odd field having a 
proper exposVfe is selected, so that a time axis shift 
of the image maV be feared. However, this does not 
pose any practical^ problem, as described with an 
example of a calculation process for a synthesized 
image shown in Fig. 4. 

s described previously, in this embodiment, each 



processes performed in the unit of two fields. A line 
interpolat^n process may be interchanged between the 
odd and even X^^l^s to make compatible with interlacing 
and reduce so-called field interference. In order to 
reduce this field oSnterf erence, a line interpolation 
circuit is inserted at a point Q between the second 
field memory 204b and stelector 207 . 

The invention is applicable to any modification or 
alteration of the above-described embodiment without 
departing from the scope and spirit thereof. 
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The present invention may be applied to a system 
constituted by a plurality of apparatuses (e,g,, a host 
computer, interface units, a reader, a printer, and the 
like) or to a system constituted by a single apparatus. 
5 The scope of the invention also includes a system 

or apparatus whose computer (CPU or MPU) runs to 
operate various devices connected thereto in accordance 
with software program codes supplied to the system or 
apparatus so as to realize the functions of the above 
10 embodiments. 

this case, the software program codes 



-v\ themselvtes realize the embodiment functions. 
III ^Therefore, Nthe program code themselves and means for 

supplying sucTk program codes to a computer, e,g- , a 
m 15 storage medium scoring such program codes, constitute 

the present inventiOT 

The storage medium storing such program codes may 
be a floppy disk, a hard disk, an optical disk, a 
magneto-optical disk, a CD-ROM, a magnetic tape, a non- 
20 volatile memory card, and a ROM. 

Obviously, such program codes are other types of 
embodiments of this invention, not only for the case 
wherein the embodiment functions are realized by 
executing the program codes supplied to the computer 
25 but also for the case wherein the embodiment functions 
are realized by the program codes used with an OS 
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(operating system) on which the computer runs or with 
other various types of application software. 

Furthermore, the scope of the invention also 
includes obviously the case wherein in accordance with 
5 the program codes stored in a memory of a function 
expansion board or unit connected to the computer 
supplied with the program codes, a CPU or the like of 
the function board or unit executes part or the whole 
of the actual tasks for realizing the embodiment 
10 functions. 

^ As described so far, according to the present 

/ invention, a plurality of images are sequentially 
\ picked up at different exposure amounts, and an 
I apparent dynamic range of a video signal is increased 
15 I by synthesizing one image from a plurality of images. 

I In such a case, if a difference between a motion vector 
between pixels in a plurality of images and a motion 
vector between images is larger than a predetermined 
threshold value, the pixel is not used for 
20 synthesization in order to increase a substantial 

dynamic range. For example, even under rear light, not 
only the main object but also the background can have 
images at proper exposures. Even a video signal having 
an image shift caused by a motion of an object itself 
25 or hand vibration can be subjected to a practically 
usable dynamic range increasing process, to thereby 
prevent double or triple images. 
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Many widely different embodiments of the present 
invention may be constructed without departing from the 
spirit and scope of the present invention. It should 
be understood that the present invention is not limited 
to the specific embodiments described in the 
specification, except as defined in the appended 
claims . 



